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FOREWORD 


This  research  and  development  was  conducted  in  support  of  project  SF57-525-001- 
002-03.01  (Human  Factors  Engineering  Technology  in  Shipboard  Combat  Systems)  under 
the  sponsorship  of  the  Naval  Sea  Systems  Command.  The  thrust  of  this  project  is  to 
enhance  the  effectiveness  of  command  and  control  systems  through  improved  design  of 
the  human-computer  interface.  In  previous  work,  a  laboratory  simulation  system  was 
developed  to  investigate  decision  making  in  antiair  warfare  (AAW)  operations  and  to 
explore  human  capacity  limits  in  a  multiple-task  environment  (NPRDC  TRs  81-15,  82-21, 
84-39;  TN  82-26).  The  present  work  examined  models  of  human  performance  in  the  AAW* 
simulation  to  identify  operative  information  processing  strategies.  This  effort  interacted 
with  and  benefited  from  ongoing  Independent  Research  project  ZR000-01-000.022  (Models 
of  Human  Performance  with  Applications  to  Decision  Aiding). 


3.  W.  RENARD 
Commanding  Officer 


3.  W.  TWEEDDALE 
Technical  Director 
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SUMMARY 


Problem 


Antiair  warfare  (AAW)  operations  require  rapid  assessment  and  prosecution  of 
threatening  contacts.  Since  it  is  unlikely  that  the  human  operator/decision  maker  will  be 
replaced  by  automation,  there  is  a  need  to  better  understand  human  decision  strategies, 
limitations,  and  cognitive  resources  for  coping  with  high  workloads.  Such  knowledge  can 
be  used  to  develop  more  effective  decision  aids  for  joint  man-machine  problem  solving. 

Objective 

The  objective  of  this  effort  was  to  infer  human  information  processing  strategies 
from  an  operator's  overt  responses  in  a  simulated  AAW  task.  In  particular,  an  attempt 
was  made  to  determine: 

1.  The  operator's  normal  processing  mode. 

2.  Whether  an  explicit  processing  instruction  yields  evidence  that  the  instructed 
strategy  is  indeed  invoked. 

3.  Whether  increased  workload  produces  an  identifiable  shift  in  the  operative 
strategy. 

Approach 

A  chief  petty  officer  with  experience  in  AAW  operations  served  as  an  operator  in  an 
experiment  that  varied  task  load  and  strategy  instructions.  Data  were  collected  to 
support  detailed  analyses  and  mathematical  modeling  of  operator  strategies.  A  mathe¬ 
matical  model  was  developed  based  on  the  targets'  ranks  on  the  relevant  attributes  of 
range  and  speed.  Two  plausible  operator  strategies  were  tested:  (1)  a  range  strategy  that 
prescribes  "fire  at  the  closest  eligible  target"  and  (2)  a  threat  strategy  that  prescribes 
"fire  at  the  eligible  target  that  will  reach  ownship  the  soonest." 

Results 


1.  The  operator's  normal  processing  mode  was  to  use  a  range  strategy.  However, 
when  so  instructed,  the  operator  did  adopt  the  more  complex  threat  strategy. 

2.  Implementation  of  the  threat  strategy,  despite  its  optimality,  increased  own- 
ship's  vulnerability.  This  was  attributed  to  its  increased  processing  burden  and  reduced 
output  rate  vs.  that  of  the  range  strategy. 

3.  No  shifts  in  processing  strategy  were  detected  as  workload  increased,  although 
performance  generally  declined,  as  expected. 

4.  There  was  a  tendency  to  prosecute  targets  in  clusters  based  on  proximities  in 
bearing.  This  constituted  a  processing  heuristic  for  coping  with  high  workloads. 

Conclusions 


1.  A  simple  mathematical  model  based  on  the  rankings  of  the  target  stimuli  can 
provide  suitable  tests  of  the  information  processing  strategies  used  by  an  operator. 


vii 


2.  No  evidence  was  obtained  for  a  shift  in  processing  strategy  as  workload 
increased;  rather,  the  effect  of  workload  may  be  to  limit  the  depth  to  which  the  operative 
strategy  is  pursued. 


3.  .  The  strategies  used  by  "experts"  are  not  necessarily  optimal;  they  may  not  be  the 
best  guidelines  for  automated  algorithms. 

Recommendations 


It  is  recommended  that,  in  continued  exploratory  research  on  AAW  decision  making 
particular  attention  be  given  to  the  following: 

1.  Inclusion  of  additional  dimensions  of  threat,  such  as  the  lethality  of  the  opposing 

platforms'  weapons.  & 

2.  Identification  of  information  processing  strategies  in  more  complex  AAW 

iOS* 

3.  Development  and  evaluation  of  decision  aids  for  threat  assessment. 
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INTRODUCTION 


Problem 


Antiair  warfare  (AAW)  operations  require  rapid  assessment  and  prosecution  of 
threatening  contacts.  Since  it  is  unlikely  that  the  human  operator/decision  maker  will  be 
replaced  by  automation,  there  is  a  need  to  better  understand  human  decision  strategies, 
limitations,  and  cognitive  resources  for  coping  with  high  workloads.  Such  knowledge  can 
be  used  to  develop  more  effective  decision  aids  for  joint  man-machine  problem  solving. 

Background 

2  The  nature  of  today's  threat  and  the  sophistication  of  modern  command  and  control 
(C  )  systems  impose  heavy  burdens  on  decision  makers.  While  innovations  in  automation 
(e.g.,  in  the  Aegis  combat  system)  address  the  demands  of  high-density  AAW  battles, 
these  do  not  obviate  human  decisions.  Rather,  operators  are  forced  to  process  more 

information  with  less  time  and  error  tolerance  than  ever  before.  Thus,  designers  of  C2 
systems  and  associated  decision  aids  must  consider  the  cognitive  skills  of  human 
operators.  Data  are  required  on  the  kinds  of  decision  strategies  they  invoke,  their 
limitations  in  processing  information,  and  the  effects  of  increasing  workload  demands. 
Computer  support  may  then  be  applied  more  appropriately  to  enhance  the  decision-making 
process. 


Only  sparse  data  are  available  on  decision-making  processes  in  C2  threat  assessment 
tasks.  Rigney  and  Debow  (1967)  reported  a  multidimensional  scaling  analysis  of  decision 
strategies  in  AAW  threat  evaluation.  They  found  that  experts  judged  threat  primarily  on 
target  range  and,  to  a  much  lesser  extent,  on  course.  Also,  they  found  that  other  factors, 
such  as  speed,  altitude,  and  composition  of  the  raid,  had  negligible  influence.  Since  the 
results  were  based  on  static  displays,  it  is  of  interest  to  determine  if  similar  results  would 
occur  in  more  dynamic  scenarios. 

A  common  finding  is  that,  when  information  is  presented  at  excessive  rates,  human 
performance  does  not  drop  catastrophically;  rather,  it  degrades  gradually  (Norman  & 
Bobrow,  1975)  as  the  decision  maker  presumably  invokes  various  "coping"  strategies 
(Sheridan  &  Ferrell,  1974).  People  apparently  adapt  by  processing  less  input  information 
and  considering  fewer  decision  alternatives  as  workload  increases  (Greitzer,  Hutchins,  & 
Kelly,  1984;  Kelly  &  Greitzer,  1982;  Serfaty,  Soulsby,  &  Kleinman,  1984).  Thus,  a 
complex  rule  may  be  replaced  by  a  simpler  decision  heuristic  if  the  operator  is 
overloaded. 

Objective 

The  objective  of  this  effort  was  to  infer  human  information  processing  strategies 
from  an  operator's  overt  responses  in  a  simulated  AAW  task.  In  particular,  attempts  were 
made  to  determine: 

1.  The  operator's  normal  processing  mode. 

2.  Whether  an  explicit  processing  instruction  yields  evidence  that  the  instructed 
strategy  is  indeed  invoked. 

3.  Whether  increased  workload  produces  an  identifiable  shift  in  the  operative 
strategy. 
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APPROACH 


AAW  Simulation 


The  present  analysis  used  a  simulated  AAW  task  (Hershman  &  Greitzer,  1982)  in 
which  an  operator  must  defend  "ownship"  by  launching  missiles  against  a  raid  of  incoming 
targets.  While  the  ultimate  objective  is  to  intercept  all  targets  at  maximum  range, 
limitations  in  human  processing  preclude  such  performance,  except  for  trivially  few 
targets.  More  realistic,  perhaps,  is  an  operational  goal  that  seeks  to  minimize  ownship's 
vulnerability,  or  total  exposure  to  threat. 

All  targets  in  the  simulation  have  equally  lethal  weapons.  Thus,  it  is  reasonable  to 
adopt  a  ship  vulnerability  function  that  increases  nonlinearly  as  a  target  approaches,  so 
that  the  most  immediate  threats  should  always  be  prosecuted  first.  In  particular,  let  the 
momentary  vulnerability  induced  by  an  eligible  (i.e.,  engageable)  target  equal  l/(time 
remaining),  where  the  time  remaining  is  measured  as  display  updates  until  the  target 
would  penetrate  and  score  a  hit  against  ownship.1  The  total  vulnerability  incurred  in  an 

engagement  is  then  taken  as  the  sum  of  the  momentary  vulnerabilities,  accumulated  over 
all  updates  and  over  all  targets. 

To  minimize  the  defined  vulnerability,  it  is  clear  that  the  operator  should  adopt  the 
following  threat  strategy:  "Always  fire  at  the  target  that  will  reach  ownship  the  soonest." 
An  alternative  prescription  is  a  range  strategy  that  says:  "Always  fire  at  the  closest 
eligible  target."  To  the  extent  that  range  and  vulnerability  are  correlated,  this  may  be  an 
attractive  compromise  or  "satisficing"  (Simon,  1957)  option  for  the  operator.  Also,  it 
nearly  meets  the  goal  of  maximizing  the  size  of  the  "free  and  clear"  zone  around  ownship. 

A  third  possible  goal  is  the  maximization  of  the  average  range-at-intercept.  This 
goal  is  met  by  a  speed  strategy  that  says:  "Fire  at  any  eligible  fast  target;  if  none  exist, 
then  fire  at  any  eligible  medium  speed  target;  if  none  exist,  then  fire  at  any  eligible  slow 
target.'  For  any  limited  capacity  processor,  this  would  produce  an  inordinate  number  of 
hits  on  ownship.  Past  results  (Kelly  <5c  Greitzer,  1982)  render  this  strategy  very  unlikely 
and  it  is  not  examined  further. 

If  one  adopts  the  goal  of  minimizing  vulnerability,  the  threat  strategy  is  optimal. 
However,  this  strategy  compels  the  operator  to  integrate  both  target  range  and  speed  in 
establishing  firing  priorities.  The  range  strategy,  on  the  other  hand,  is  not  directed  at 
vulnerability  per  se,  but  it  requires  only  one  target  attribute  (namely,  range)  for  its 
implementation.  Thus,  the  operator's  processing  rate  should  be  higher  for  the  simpler 
range  strategy. 

In  the  face  of  high  task  demands,  a  switch  from  a  threat  strategy  to  a  range  strategy 
may  incur  little  or  no  loss  in  decision  quality,  as  measured  by  ownship's  vulnerability. 
Indeed,  vulnerability  may  be  reduced  if  output  is  sufficiently  increased  over  that  of  the 
more  complex  strategy.  Greitzer,  Hutchins,  and  Kelly  (1984)  could  not  reliably  discrimi¬ 
nate  in  their  data  a  range-based  strategy  from  a  threat-based  one.  The  present  effort 
pursues  this  work  by  formulating  and  testing  mathematical  models  of  operator  strategies. 


In  general,  it  is  sufficient  to  assume  any  decreasing,  concave  upward  function;  that 
is,  the  function's  first  derivative  must  be  <  0  and  its  second  derivative  must  be  >  0. 
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Procedure 


The  task  (see  Hershman  6c  Greitzer,  1982  for  details)  was  run  by  a  Tektronix  4054A 
microcomputer,  which  simulates  a  Navy  Tactical  Data  System  (NTDS)  display  on  which 
hostile  air  targets  approach  ownship.  In  each  engagement  of  approximately  4  minutes,  the 
raiding  targets  approach  at  one  of  three  speeds,  on  constant  courses,  and  from  random 
directions. 

The  operator  fires  a  missile  by  entering  a  target's  two-digit  track  number  on  the 
computer  keyboard.  The  computer  updates  the  display  every  2  seconds.  Manipulation  of 
the  number  of  targets  and  their  arrival  rates  during  the  middle  2  minutes  of  an 
engagement  serves  to  produce  various  track  loads  on  the  operator.  Data  are  stored  on 
floppy  diskette  for  later  analysis  and  reconstruction  of  operator  performance. 

A  35-year-old  chief  petty  officer  with  experience  as  an  air  intercept  controller  and 
anti-submarine  air  controller  served  as  an  operator.  He  had  been  trained  earlier  on  the 
experimental  task. 

Design 

Two  factors  were  manipulated  in  a  2x2  experimental  design:  strategy  instructions 
and  workload.  In  the  NORMAL  instruction  condition,  the  operator  was  asked  to  perform 
as  he  normally  would  in  prosecuting  the  targets.  In  the  THREAT  condition,  he  was 
explicitly  instructed  to  minimize  threat;  viz.,  always  try  to  prosecute  the  target  that  will 
hit  ownship  the  earliest.  The  THREAT  condition  was  a  departure  from  the  operator's 
normal  strategy,  but  he  reported  no  difficulty  in  adopting  it. 

Workload  was  manipulated  by  using  two  target  arrival  rates:  a  moderate  rate  of  0.25 
targets  per  sec  (yielding  a  total  of  30  targets)  and  a  high  rate  of  0.4  targets  per  sec  (with 
a  total  of  48  targets).  There  were  30  engagements  in  each  of  the  instruction  conditions. 
Nineteen  of  the  NORMAL  engagements  and  18  of  the  THREAT  engagements  had  a  0.4 
target  rate.  Two  blocks  of  engagements  (NORMAL  and  THREAT)  were  run  on  each  of  3 
days  to  complete  the  data  collection. 


RESULTS 


Data  Analysis 

Figure  1  shows  the  means  of  four  performance  measures— the  skill  rating2  (100  is 
perfect),  range-at-intercept  (21  miles  is  perfect),  launch  rate,  and  vulnerability- for  the 
last  ten  engagements  in  each  of  the  four  experimental  conditions.  Two-factor  analyses  of 
variance  showed  that  the  effect  of  increased  workload  was  significant  (p  <  .01)  on  each 
measure;  that  is,  the  0.4  target  rate  reduced  the  skill  rating  and  range-at-intercept  and 
increased  the  launch  rate  and  ship's  vulnerability.  Compared  with  the  NORMAL  condi¬ 
tion,  the  THREAT  instruction  significantly  reduced  launch  rate  (p  <  .051  and  increased 
vulnerability  (p  <  .001);  effects  on  the  other  two  measures  were  marginal  (.05  <  p  <  .10). 
Since  launch  rate  showed  a  ceiling  effect  at  low  workloads,  a  separate  analysis  was  made 


2 The  skill  rating  is  defined  as  R  =  100  (average  range-at-intercept)-  12  (no.  of  hits)-2 
(no.  of  inflight  launches).  An  inflight  launch  is  any  firing  at  an  engaged  target,  and  is 
counted  as  an  operator  error. 
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on  the  high  workload  data  in  panel  (c);  it  showed  that  the  THREAT  instruction  signifi¬ 
cantly  reduced  the  launch  rate  in  the  0.4  target  rate  condition  (p  <  .05).  The  THREAT 
strategy  also  increased  ship's  vulnerability  under  high  workload,  as  the  interaction  in  panel 
(d)  was  significant  (p  <  .001).  These  findings  suggest  that  the  THREAT  condition  posed  a 
greater  processing  burden  for  the  operator  than  did  his  NORMAL  strategy. 
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Figure  1.  Four  performance  measures  as  functions  of  target  rate 
and  instruction  condition:  (a)  skill  rating,  (b)  range-at 
intercept,  (c)  launch  rate,  and  (d)  ownship  vulnerability. 
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One  reason  for  the  increased  burden  imposed  by  the  THREAT  condition  is  its 
requirement  to  search  a  broader  area  of  the  display  to  find  the  most  threatening  target. 
In  contrast,  the  NORMAL  condition  might  afford  an  opportunity  to  prosecute  clusters  of 
targets  in  close  proximity  to  reduce  the  search  times.  To  test  this  notion,  analyses  were 
conducted  on  each  "burst"  of  launch  activity,  operationally  defined  as  a  string  of  launches 
in  consecutive  updates.  Figure  2  reconstructs  the  bursts  in  typical  engagements  in  the 
two  instruction  conditions.  Launches  within  a  burst  are  joined  by  line  segments  and 
indicate  the  extent  of  the  operator’s  scan  of  the  display.  The  numerals  identify  the 
starting  points  and  chronology  of  the  burst  activity. 

Table  1  shows  various  burst  statistics  for  the  0.4  target  rate  conditions.  As 
hypothesized  above,  a  clustering  tactic  should  produce  fewer  and  larger  bursts  in  the 
NORMAL  condition,  but  these  differences  were  not  significant.  Another  test  for 
clustering  is  the  angular  separation  between  consecutive  intraburst  vs.  interburst 
launches.  In  particular,  clustering  should  produce  smaller  angular  separations  for  the 
former  and  larger  separations  (near  the  chance  value  of  90°)  for  the  latter.  If  clustering  is 
not  operative,  the  angular  separation  should  approximate  90  degrees  for  both  kinds  of 
launches.  These  data  support  differential  clustering  in  the  two  instruction  conditions: 
For  the  NORMAL  condition,  the  mean  intraburst  separation  was  significantly  less 
(p  <  .001)  then  the  interburst  separation,  which  did  not  differ  reliably  from  90  degrees. 
For  the  THREAT  condition,  the  mean  angles  differed  reliably  (p  <  .01),  but  neither  angle 
differed  statistically  from  90  degrees.  Thus,  there  was  a  reduced  tendency  to  cluster 
targets  in  the  THREAT  condition. 

Mathematical  Models 


To  elucidate  possible  strategies,  consider  the  eligible  targets  (say,  k  in  number)  that 
might  be  present  on  any  update.  Then  rank  the  k  targets  based  on  a  presumed  strategy. 
For  example,  a  range  strategy  would  assign  rank  =  1  to  the  closest  target  and  rank  =  k  to 
the  most  distant  one,  etc.  A  threat  strategy  would  give  rank  1  to  the  target  that  will  hit 
ownship  the  soonest,  etc.  As  the  operator  executes  a  given  strategy,  his  observed 
launches  can  be  expected  to  conform  generally  to  the  rankings  imposed  by  the  operative 
strategy. 

To  be  more  precise,  the  approach  is  as  follows:  Let  p.(k)  denote  the  probability  of 

firing  at  the  jth-ranked  (based  on  any  presumed  strategy)  of  k  eligible  targets.  First,  a 
two-parameter  model  is  developed  to  predict  the  observed  {pj(k)};  that  is,  the  probabili¬ 
ties  of  choosing  the  best  of  the  eligible  targets  for  observed  values  of  k  =  2,3,...  16.  This 
affords  preliminary  testing  of  the  range  and  threat  strategies.  Then,  without  estimating 
additional  parameters,  predictions  are  made  for  each  of  the  p.(k)  distributions  (a  total  of 
135  points).  1 

The  observed  {p^k)}  are  graphed  as  the  open  circles  in  Figure  3.  Figures  3a  and  3b 

are  for  the  NORMAL  instruction  condition  based  on  range  and  threat  rankings  respec¬ 
tively;  and  Figures  3c  and  3d,  for  the  THREAT  instruction.  (Data  for  the  two  target  rates 
were  combined.)  If  a  given  strategy  is  operative,  then,  as  a  minimum  first  test,  the 
observed  { p j  (k)}  based  on  that  strategy  generally  should  decrease  as  k  increases.  As  seen 

in  Figure  3,  both  the  proposed  range  and  threat  strategies  meet  this  test  in  both 
instruction  conditions.3 


3  Fewer  observations  were  available  for  larger  k,  making  these  data  less  reliable. 
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Table  1 


Target  Clustering  Statistics 


Instruction  Condition 

Item 

NORMAL 

THREAT 

Number  of  Bursts 

14.1 

15.4 

Size  of  Bursts 

4.1 

3.6 

Degrees  Separation: 

Intraburst 

Interburst 

55.2 

91.1 

74.5 

88.6 

Note.  Target  rate  =  0.4. 


As  a  second  test,  if  the  operator  is  executing  the  proposed  strategy,  then  the 
observed  {p^k)}  should  each  exceed  1/k,  the  values  predicted  by  a  random  "strategy"  (the 

dashed  curves  in  Figure  3).  This  holds  in  Figure  3a  but  not  in  Figure  3b,  in  which  the 
probability  of  firing  at  the  most  threatening  of  only  two  targets  is  less  than  chance.  Thus, 
data  in  the  NORMAL  condition  tentatively  support  a  range  strategy  as  opposed  to  a  threat 
strategy.  Similarly,  in  the  THREAT  condition  of  Figures  3c  and  d,  the  data  suggest  the 
employment  of  a  threat  strategy.  Note  that  the  observed  {Pj(k)}  do  not  approach  zero,  as 

one  might  expect.  Indeed,  the  likelihood  of  choosing  the  best  of  10-16  eligible  targets 
(see  footnote  3)  is  about  the  same  as  choosing  the  best  of  5.  A  possible  explanation  is 
that,  as  the  observer  searches  the  display  for  high  priority  targets,  many  of  those  in  the 
periphery  (at  long  range)  can  be  eliminated.  Actually,  to  execute  either  strategy,  only 
three  targets  need  be  considered:  the  closest  fast,  medium,  and  slow  targets.  A 
restricted  search,  generally  near  ownship,  will  likely  yield  a  small  set  that  contains  the 
"promising"  targets.  Given  enough  time,  the  best  target  in  this  restricted  set  can 
certainly  be  determined,  but  the  advantage  gained  must  be  traded  off  against  search  time. 
In  general,  as  the  target  density  increases,  the  demand  to  "keep  up"  forces  the  operator  to 
spend  less  time  resolving  fine  distinctions  among  several  good  targets.  Thus,  for  suitably 
large  k,  the  choice  may  be  random  among  a  restricted  set  of,  say,  3-5  targets. 
Presumably,  target  density  compels  a  time  pressure  that  governs  the  rate  at  which  this 
choice  approaches  randomness.  Equation  (1)  captures  this  interpretation: 

Pj(k)  =  b  +  (1-b)  exp{-a(k-l)},  a,b>0.  (1) 

The  parameter  b  is  the  chance  selection  probability  in  the  restricted  set  of  targets 
so  that  1/b  estimates  the  size  of  this  set.  Based  on  human  information  processing 
limitations  (e.g.,  Miller,  1956),  one  might  expect  the  set  size  to  be  about  7,  but  3-5  seems 
more  appropriate  for  this  task.  This  should  yield  a  value  for  b  in  the  range  of  0.20-0.33. 
The  parameter  a  represents  the  decision  maker's  sensitivity  to  target  density1*  by 


Alexandridis,  Entis,  Wohl,  and  Deckert  (1984)  used  a  similar  exponential  decay 
function  to  model  the  effect  of  workload  on  the  optimality  of  an  antisubmarine  warfare 
commander's  tactical  decisions. 
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d.  THREAT  Condition. 


—  Threat  model. 
Randcra  model. 

Parameter  a  =  0.41 
Parameter  b  =  0.30 
Squared  error  =  239.16 


Figure  3.  Probability  of  engaging  the  first-ranked  of  k  eligible 
targets  as  a  function  of  instruction  condition  and 
assumed  model.  Observed  data  are  plotted  as  open 
circles. 
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determining  the  randomness  of  the  choice  of  a  target  in  the  restricted  set;  that  is,  a 
controls  the  rate  at  which  Pj(k)  approaches  its  asymptote,  b.  For  example,  a  =  0  implies 

complete  persistence  in  processing  the  targets  so  that  the  first-ranked  target  would 
always  be  selected,  as  if  there  were  no  time  pressure  at  all.  As  a  increases,  there  is  less 
dedication  to  the  processing  strategy;  the  operator  is  willing  to  compromise  his  goal, 
presumably  to  gain  time. 

Equation  (1)  was  fit  to  the  observed  P[(k)  distributions  for  the  two  instruction 

conditions,  assuming  each  of  the  two  strategies.  Returning  to  Figure  3,  these  predictions 
are  plotted  as  solid  lines.  The  values  of  a  and  b  that  minimize  the  squared  error* * * 5  and  the 
obtained  square  error  are  also  shown. 

In  the  NORMAL  condition,  the  range  model  provided  a  better  fit  to  the  p  (k)  data, 

with  a  =  0.72  and  b  =  0.28.  The  number  of  targets  in  the  restricted  set  is  estimated  at 
1/b  -  3.6.  In  the  THREAT  condition,  the  range  model  again  yielded  the  lower  squared 
error,  with  best-fitting  parameter  values  a  =  1.13  and  b  =  0.24.  However,  as  previously 
noted,  the  data  in  Figure  3c  do  not  favor  a  range  strategy;  a  decision  about  the  operator's 
strategy  must  be  deferred  until  the  entire  probability  distributions  are  analyzed.  The 
threat  model  yielded  the  least  squares  solution  a  =  0.41,  b  =  0.30;  in  this  case,  the  size  of 
the  restricted  set  is  estimated  at  3.3. 

Equation  (1)  predicts  only  the  likelihood  that  the  best  of  k  targets  be  engaged; 
however,  the  observer's  operative  strategy  should  also  be  manifest  in  the  relative 
frequency  with  which  the  lower-ranked  targets  are  prosecuted.  In  particular,  consider  an 
engagement  of  the  second-ranked  of  k  targets.  It  is  proposed  that  this  event  occurs  only 
if  the  first-ranked  is  not  selected  (perhaps  overlooked,  processed  with  error,  etc.),  and 
then  the  best  of  the  remaining  (k-1)  targets  is  engaged.  Thus,  the  prediction  is  that 

p2(k)  =  { 1-pj (k)}  Pj(k-l). 

Extending  this  simple  notation  to  the  probability  p.(k)  that  the  jth-ranked  target  is  the  one 
fired  at,  the  result  is:  > 


Pj(k)  =  Pl(k  -j+1)  {l-Pj(k  -i+1)},  j>l  (2) 

The  model  thus  makes  predictions  for  j=2,...k  based  only  on  the  { p ,  (k)},  for  which  Equation 
(1)  used  15  data  points  to  estimate  the  two  parameters,  a  and  b.  1 

Tests  of  Models 


Figures  4  and  5  show,  for  each  instruction  condition  and  presumed  strategy  (range  or 

threat),  the  observed  data  (open  circles),  the  ensemble  of  135  predictions  for  k=2  to  16, 

j=1  t0  k  (solid  curves),  and  predictions  for  a  strategy  that  fires  at  random  (dashed  lines).’ 


5  Observed  and  predicted  frequencies  (rather  than  probabilities)  were  used  in  all  of 

the  least-squares  analyses;  that  is,  the  prediction  errors  were  weighted  by  the  number  of 
cases  available  for  each  k. 
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b.  THREAT  instruction  condition. 


Figure  4.  RANGE  model  predictions  ( - ),  RANDOM  model  pre¬ 
dictions  ( - ),  and  observed  probabilities  of  firing  (o)  at 

the  jth  closest  target.  Number  of  targets  (k)  =  2  to  16. 
(Note:  Values  shown  for  err  and  rnd  =  squared  errors  for 
range  and  random  models  respectively.) 
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b.  THREAT  instruction  condition. 

Figure  5.  THREAT  model  predictions  ( - ),  RANDOM  model 

predictions  (  ),  and  observed  probabilities  of  firing  (o) 

at  the  jth  most-threatening  target.  Number  of  targets 
(k)  =  2  to  16.  (Note:  Values  shown  for  err  and  rnd  = 
squared  errors  for  THREAT  and  RANDOM  models 
respectively.) 
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For  the  latter,  the  probability  that  the  jth  ranked  of  the  targets  is  the  one  fired  at  is 
simply  1/k.  Although  the  same  engagements  were  used  for  testing,  the  range  and  threat 
models  were  fit  to  different  probability  distributions;  namely,  those  induced  by  the  two 
presumed  target  rankings.  Simple  comparisons  of  goodness  of  fit  may  therefore  be 
misleading.  It  is  perhaps  more  revealing  to  compare  each  model's  error  relative  to  the 
random  model’s  error,  when  predicting  the  same  data.  Results  are  summarized  in  Table  2 
and  discussed  below.  (Note  that  the  RANGE  and  THREAT  models  refer  to  the  ranking 
model  of  equation  (1),  as  based  on  the  range  and  threat  ranks  respectively.) 

!•  For  the  NORMAL  condition,  the  squared  error  for  the  range  model  was  only  13 
percent  of  that  for  the  random  model;  the  comparable  error  ratio  for  the  threat  model 
was  67  percent.  Thus,  the  range  model  was  a  better  predictor  of  performance  in  the 
NORMAL  condition.  Inspection  of  Figures  4a  and  5a  reinforces  this  finding.  The  fit  of 
the  range  model  in  Figure  4a  is  qualitatively  more  satisfying  than  that  of  the  threat  model 
in  Figure  5a.  It  appears  that  the  range  strategy  is  a  reasonable  description  of  the 
operator's  normal  processing  mode. 

2.  For  the  THREAT  instruction  condition,  the  squared  error  for  the  threat  model 
was  only  12  percent  of  that  for  the  random  model;  the  comparable  error  ratio  for  the 
range  model  was  70  percent.  Thus,  the  threat  model  was  the  better  predictor  in  the 
THREAT  condition.  This  is  also  seen  in  the  superior  fits  of  Figure  5b  vs.  those  in  Figure 
4b.  It  is  reasonable  to  conclude  that  the  operator  was  able  to  adopt  a  threat  strategy 
when  instructed  to  do  so.  -  6 


Table  2 


Errors  in  Predicting  the  p.(k)  Distributions 


NORMAL  Condition 

THREAT  Condition 

Item 

Range 

Threat 

Range 

Threat 

Model 

Model 

Model 

Model 

Strategy  Model: 

Squared  error 

906.89 

2,701.18 

1,726.82 

1,317.27 

Average  error 

0.83 

1.43 

1.21 

1.06 

Random  Model: 

Squared  error 

Average  error 

7,130.69 

2.32 

4,004.69 

1.74 

2,483.45 

1.45 

10,547.45 

2.99 

Ratio3 

0.13 

0.67 

0.70 

0.12 

Number  of  cases*3 

1,319 

1,176 

^  '  ‘  .  " 

Strategy  model  squared  error/random  model  squared  error. 

Total  number  of  display  updates  in  which  2-16  targets  were  eligible  and  a  launch  was 
made. 
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Test  for  Strategy  Shift 


To  determine  whether  increased  workload  produced  an  identifiable  shift  in  the 
operators  strategy,  the  0.25  and  0.40  target  rate  conditions  were  analyzed  separately. 
The  result  was  that  the  relative  fits  of  the  models  were  the  same  as  for  the  combined 
data  reported  above.  Thus,  there  was  no  gross  change  in  strategy  for  moderate  vs.  high 
workload  engagements. 

A  more  detailed  test  for  strategy  shifts  was  made  by  separately  analyzing  perfor¬ 
mance  for  low  load  displays  (2-5  eligible  targets)  and  high  load  displays  (10-16  eligible 
targets),  regardless  of  the  actual  target  rate.  A  shift  in  strategy  from  threat  to  range 
would  be  most  likely  in  the  THREAT  instruction  condition.  Here,  the  average  error  for 
the  threat  model  increased  from  1.04  at  low  load  to  only  1.10  at  high  load — hardly  an 
abandonment  of  the  threat  strategy.  The  average  error  of  the  range  model  increased 
from  1.09  at  low  load  to  1.25  at  high  load;  a  decrease  would  be  expected  if  there  was  a 
switch  to  the  range  strategy.  In  the  NORMAL  condition,  the  error  for  each  model  also 
increased.  Thus,  while  there  was  some  degradation  in  executing  the  strategies  at  high 
loads,  there  was  no  evidence  for  a  shift  in  strategy. 


DISCUSSION 

Detailed  data  collection,  together  with  a  rather  simple  mathematical  model  (based 
only  on  rank-order  information),  proved  useful  in  making  inferences  about  an  operator's 
threat  assessment  strategies.  A  random  engagement  strategy  was  clearly  rejected.  A 
range  strategy  reasonably  described  the  operator's  normal  processing  mode  and  reinforces, 
in  an  interactive  and  dynamic  task,  the  earlier  results  of  Rigney  and  DeBow  (1967). 

When  instructed  to  do  so,  the  operator  gave  ample  evidence  of  adopting  the  optimal 
but  more  complex  threat  strategy.  However,  it  resulted  in  poorer  performance  (increas¬ 
ing  ship  s  vulnerability),  since  it  could  not  be  executed  as  rapidly  as  the  nonoptimal  range 
strategy.  The  operator's  preference  for  the  simpler  range  strategy  can  be  interpreted  as 
an  optimal  use  of  nonoptimal  resources— a  good  example  of  well-advised  coping. 

Of  course,  it  is  preferable  that  the  proper  strategy  be  executed  at  a  high  rate.  This 
is  precisely  the  goal  of  so-called  automated  subsystems  (e.g.,  those  in  Aegis),  but  even 
these  depend  on  human  monitoring  and  intervention  (RCA,  1984).  Thus,  operators  will 
continue  to  require  supplemental  displays  and  aids.  In  particular,  there  is  a  need  for  a 
display  format  that  codes  threat  directly;  that  is,  without  demanding  complex  cognitive 
transformations  by  the  user.  Barnes  (1983)  has  compared  two  such  formats  in  the  context 
of  airborne  electronic  warfare. 

Regarding  possible  strategy  shifts,  none  was  detected  either  between  low  vs.  high 
workload  engagements  or  between  low  vs.  high  density  displays  within  engagements.  This 
is  not  to  say  that  the  operator  rigidly  executes  his  adopted  strategy.  Indeed,  adaptability 
to  task  difficulty  is  evident  in  the  observed  p^k)  distributions  (Figure  3),  and  has 

prominence  in  the  model  via  the  parameter  a  in  Equation  (1).  To  the  extent  that  the 
model  accurately  reflects  human  performance,  it  suggests  that  the  operator  spends  less 
and  less  time  in  search  and  decision  processes  as  target  density  increases. 

Note  that  the  values  obtained  for  the  parameter  a  imply  more  persistence  in  using 
the  prescribed  threat  strategy  (a  =  0.41)  then  the  unprompted  range  strategy  (a  =  0.72) 
This  seems  inconsistent  with  the  difficulty  of  the  former,  as  it  should  degenerate  toward 
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randomness  more  quickly  and  yield  thereby  a  larger  value  of  a.  However,  the  explicit 
instruction  to  use  a  threat  strategy  may  have  induced  its  greater  persistence.  The 
obtained  values  of  b  yield  estimates  of  3.0-3.3  for  the  size  of  the  restricted  set  of  targets, 
consistent  with  the  expectation  based  on  human  information  processing  limits. 

It  is  worth  remarking  that  no  "pure"  strategies  were  observed;  significant  target 
clustering  was  generally  the  case.  This  tendency  was  strongest  in  the  NORMAL  condition, 
in  which  the  operator  used  a  range  strategy  tempered  by  clustering  of  targets  based  on 
bearings.  While  clustering  incurs  a  cost  (the  increased  likelihood  of  overlooking  a  better 
target),  the  higher  processing  rate  that  it  affords  makes  it  a  major  heuristic  for  coping 
with  high  workload  demand.  In  this  regard,  a  standard  practice  in  NTDS  A  AW  operations 
is  one  of  "sectored  responsibility";  that  is,  an  individual  operator  is  assigned  a  single 
bearing  sector  of  90-120  degrees  in  extent.  This  division  of  labor  obviously  reduces  the 
adverse  effects  of  an  operator's  tendency  to  cluster  targets  based  on  proximities  in 
bearing.  The  current  results  thus  support  the  NTDS  "sectoring"  practice. 


CONCLUSIONS 

1*  It  was  demonstrated  that  a  simple  mathematical  model  based  on  the  rankings  of 
the  target  stimuli  can  provide  suitable  tests  of  the  information  processing  strategies  used 
by  an  operator. 

2.  No  evidence  was  obtained  for  a  shift  in  processing  strategy  as  workload 
increased;  rather,  the  effect  of  workload  may  be  to  limit  the  depth  to  which  the  operative 
strategy  is  pursued. 

3.  The  strategies  used  by  "experts"  are  not  necessarily  optimal;  they  may  not  be  the 
best  guidelines  for  automated  algorithms. 


RECOMMENDATIONS 

It  is  recommended  that,  in  continued  exploratory  research  on  AAW  decision  making, 
particular  attention  be  given  to  the  following: 

1.  Inclusion  of  additional  dimensions  of  threat,  such  as  the  lethality  of  the  opposing 
platforms'  weapons. 

2.  Identification  of  information  processing  strategies  in  more  complex  AAW 
scenarios. 


3.  Development  and  evaluation  of  decision  aids  for  threat  assessment. 
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